Tag: power smoothing
2 reviews
Smoothing the power curve of LLM training and inference
Which kernel- and scheduler-level techniques reduce power ripples during LLM training and inference, and at what cost in energy and throughput?
GPU power draw during LLM training and inference is bursty: compute-bound prefill spikes, memory-bound decode idles far below the power ceiling, and synchronised training steps add further lumps. Across 89 sources (2011-2026) the evidence shows GPU frequency and power-capping settings have pronounced energy sweet spots (routinely 15-35% energy saved for single-digit throughput loss) that apply only to compute-bound phases; the newest work shows the power ramp between scheduling steps, not the peak, is the controllable quantity. Confidence is moderate: device-level trade-offs are replicated across many groups, while the ripple-smoothing literature is young (mostly 2025-2026 preprints) and largely single-node or emulation-scale.
Energy efficiency across the AI datacenter stack
What does the 2023-2026 literature establish about improving energy efficiency across the AI datacenter stack - from GPU power management and workload scheduling to grid-level coordination - and how strong is the evidence for each lever?
AI datacenters are among the fastest-growing electricity consumers of the 2020s, and the 2023-2026 literature offers a stack of levers to improve their efficiency: GPU power capping and frequency control, energy- and carbon-aware scheduling, LLM serving configuration, cooling control, and grid-interactive operation. Evidence is strongest at the device and workload levels (measured savings of roughly 15-50% on specific workloads), moderate for scheduling (mostly simulation), and thinnest at grid level, where a single production deployment carries much of the weight. The key caveat: nearly every lever is workload- and hardware-generation-specific - power capping that saves ~23% energy on one model hurts SLOs on another - so none generalises without measurement.